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International application No. 


INTERNATIONAL PRELIMINARY REPORT ON PATENTABILITY 


PCT/JP2004/016454 


Box No. I Basis of the report 


1 . With regard to the language, this report is based on: 




the international application in the language in which it was filed 




fj a translation of the international application into 


, which is the language of a 


translation furnished for the purposes of: 




□ international search (Rules 1 2.3(a) and 23.1(b)) 




jH publication of the international application (Rule 1 2.4(a)) 




[H international preliminary examination (Rules 55.2(a) and/or 55.3(a)) 




2. With regard to the elements of the international application, this report is based on (replacement sheets which have been 


furnished to the receiving Office in response to an invitation under Article 14 are referred to in this report as "originally Sled" 


and are not annexed to this report): 




jH the international application as originally filed/furnished 




fy? the description: 




pages 1 - 3 , 5 - 1 8 


as originally filed/furnished 


pages* 4,4/1 received by this Authority on 21.11.2005 


pages* received by this Authority on 


the claims: 




Nos. 3-8, 11-12 


as originally filed/furnished 


Nos.* as amended (together with any statement) under Article 19 


Nos.* 1,9,13-16 received by this Authority on 21.11.2005 


Nos.* received by this Authority on 


jf? the drawings: 




sheets/6ge 1-6 


as originally filed/furnished 


sheets/figs * received by this Authority on 


sheets/figs * received by this Authority on 


jH a sequence listing and/or any related table(s) - see Supplemental Box Relating to Sequence Listing. 


3. j>7: The amendments have resulted in the cancellation of: 




|H the description, pages 




ffi the claims, Nos. 2,10 


jH the drawings , sheets/figs 


jH the sequence listing (specify)'. 




JH any table(s) related to sequence listing (specify): 




4. PI This report has been established as if (some of) the amendments annexed to this report and listed below had not been 


made, since they have been considered to go beyond the disclosure as filed, as indicated in the Supplemental Box 


(Rule 70.2(c)). 




jH: the description, pages 




jH the claims, Nos. 


jH the drawings, sheets/figs 


(H the sequence listing (specify/ 




JH any table(s) related to sequence listing (specify): 




* If item 4 applies, some or all of those sheets may be marked "superseded. " 
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Box No. V Reasoned statement under Article 35(2) with regard to novelty, inventive step or industrial applicability; 
citations and explanations supporting such statement 



1 . Statement 

Novelty (N) Claims 1, 3-9, 11-16 . YES 

Claims NO 

Inventive step (IS) Claims YES 

Claims 1, 3~9, 11-16 NO 

Industrial applicability (IA) Claims 1, 3-9, 11-16 YES 

Claims NO 



2. Citations and explanations(Rule 70.7) 

D1:JP 2001-187552 A (KABUSHIKI KAISYA TOYOTA JIDOUSHOKKI SEISAKUSYO) 
2001.07.10 . 

D2:JP 2004-004043 A (MATSUSHITA DENKISANGYOU KABUSHIKI KAISYA) 
2004.01.08 

D3:JP 2001-285681 A (MATSUSHITA DENKISANGYOU KABUSHIKI KAISYA) 
2001.10.12 

(claiml, 3-9, 11-16) 

The technical feature [VIDEO IMAGE POSITIONAL RELATIONSHIP 
CORRECTION APPARATUS] in Dl,the feature [ CAMERA CORRECTOR] in D2 and 
the feature [CALIBRATION SYSTEM] in D3 are concerned with mutually 
related technical fields. 

Therefore, the skilled person in the art would easily conceive the 
idea of employing the feature disclosed in lines 24-50 page 23, lines 
01-29 page24 in D2 or in lines 44-50 column 9, lines 35-37 column 10 
in D3 to combine the feature in Dl . 

(claiml, 9) 

Dl discloses [the recognition means providing a virtual target and 
carrying out the recognition] (see lines 26-50 column 5, lines 1-21 
column 6) . 

The skilled person would regards it as a design procedure to employ 
the technical feature [Values of the coordinate conversion parameters 
before modification are used as initial values of the coordinate 
conversion parameters] . Therefore the person skilled in the art would 
easily conceive the idea of employing the feature [a virtual target 
is one produced based on coordinate conversion parameters before 
modification] . 
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Supplemental Box 



In case the space in any of the preceding boxes is not sufficient. 

Continuation of: Box No. V 

(claiml3, 15) 

Dl discloses [the recognition means automatically recognizes the 
nearest actual target] (see lines 3-8 column 6). 

The skilled person would regard it as a normal option not. to display 
the virtual target. 

(claiml4, 16) 

Dl discloses [lines extending between new virtual targets] (see lines 
36-44 columnl3,fig 9). 

The skilled person would regard it as a design procedure to employ 
the feature [lines extending between new virtual targets based on the 
coordinate conversion parameters after modification are displayed] . 
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of the relational expressions is larger than the number of the 
coordinate conversion parameters which require correction; the 
recognition means providing a virtual target in the monitor 
coordinate system on the monitor screen based on the coordinate 
conversion parameters before modification using the coordinate 
conversion means, and carrying out the recognition based on the 
difference between the monitor coordinate of the image of the actual 
target captured actually by the camera and the monitor coordinate 
of the virtual target. 

According to the present invention, a steering assist apparatus 
includes the above video image positional relationship correction 
apparatus, in which the actual video image and the virtual video image 
are a video image at the back of the vehicle, and a steering assist 
guide , respectively. 

Further, according to the present invention, a method of correcting 

relative positional relationship between an actual video image 
captured by a camera and a virtual video image when superimposing 
the actual image and the virtual video image on a monitor screen, 

includes the steps of: capturing actual targets in an actual 
coordinate system by the camera; theoretically deriving monitor 
coordinates in a monitor coordinate system on the monitor screen by 
coordinate conversion of actual coordinates of the actual targets 

in the actual coordinate system based on reference values of 
coordinate conversion parameters including internal parameters of 
the camera itself and attachment parameters for attaching the camera 
to the vehicle; providing a virtual target in the monitor coordinate 
system on the monitor screen based on the coordinate conversion 

parameters before modification and recognizing the monitor 
coordinates of the image of the actual targets actually captured by 
the camera based on the difference between the monitor coordinate 
of the image of the actual target captured actually by the camera 
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and the corresponding monitor coordinate of the virtual target; 
generating relational expressions based on deviations between the 
monitor coordinates of the image of the actual targets and the monitor 
coordinates in the monitor coordinate system of the actual targets 
which have been subjected to coordinate conversion, the number of 
relational expressions being larger than the number of the coordinate 
conversion parameters to be corrected including at least internal 

parameters of the camera itself of the coordinate conversion 
parameters; correcting the coordinate conversion parameters such 
that the square-sum of the deviations is the minimum; and correcting 
relative positional relationship between the actual video image and 

the virtual video image based on the corrected values of the 
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1. (amended) An apparatus for correcting relative positional 

relationship between an actual video image captured by a camera and 
a virtual video image for use in a video image display device for 
superimposing the actual video image and the virtual video image on 
a monitor screen, comprising: 

actual targets set in an actual coordinate system in an area 
captured by the camera; 

coordinate conversion means for theoretically deriving monitor 
coordinates in a monitor coordinate system on the monitor screen by 
coordinate conversion of actual coordinates of the actual targets 
in the actual coordinate system based on reference values of 
coordinate conversion parameters including internal parameters of 
the camera itself and attachment parameters for attaching the camera 
to the vehicle; 

recognition means for recognizing the monitor coordinates of 
the image of the actual targets actually captured by the camera; 
and 

correction means for correcting at least values of the internal 
parameters of the camera itself of the coordinate conversion 
parameters based on deviations between the monitor coordinates of 
the image of the actual targets actually captured by the camera and 
the corresponding monitor coordinates in the monitor coordinate 
system of the actual targets which has been subi ected to the coordinate 
conversion, and correcting relative positional relationship between 
the actual video image and the virtual video image based on the 
corrected values of the coordinate conversion parameters, 

the correction means generating relational expressions the 
number of which is larger than the number of the coordinate conversion 
parameters based on the monitor coordinates of the image of the actual 
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targets and the monitor coordinates in the monitor coordinate system 
of the actual targets which have been subjected to coordinate 
conversion, the coordinate conversion parameters being corrected 
such that the square-sum of the deviations is the minimum; 

the number of actual targets being determined such that the 
number of the relational expressions is larger than the number of 
the coordinate conversion parameters which require correction, 

the recognition means providing a virtual target in the monitor 
coordinate system on the monitor screen based on the coordinate 
conversion parameters before modification using the coordinate 
conversion means, and carrying out the recognition based on the 
difference between the monitor coordinate of the image of the actual 
target captured actually by the camera and the monitor coordinate 
of the virtual target. 

2. (cancelled) 

3. A video image positional relationship correction 
apparatus according to claim 2, wherein the recognition means 
includes a controller for moving one of the actual target and the 
virtual target on the monitor screen to a position overlapped on the 
other of the actual target and the virtual target by manipulation 
of an operator. 

4. A video image positional relationship correction 
apparatus according to claim 3, wherein the controller includes 
direction buttons for inputting a correction amount of one of the 
actual target and the virtual target on the monitor screen in an up 
direction, a down direction, a left direction and a right direction, 
a decision button for confirming a condition in which the actual target 
and the virtual target are overlapped with each other, and a 
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calculation button for allowing the correction means to start 
correction calculation. 

5. A video image positional relationship correction 
apparatus according to claim 1, wherein the recognition means 
includes an image processing circuit for carrying out the recognition 
by image processing. 

6. A steering assist apparatus having a video image 
positional relationship correction apparatus according to claim 1, 
wherein the actual video image and the virtual video image are a video 
image at the back of the vehicle and a steering assist guide, 
respectively. 

7. A steering assist apparatus according to claim 6, 
wherein the actual target is set on a road surface. 

8. A steering assist apparatus according to claim 6, 
wherein the actual target is set on a planar member attached to a 
rear portion of the vehicle. 

9. (amended) A method of correcting relative positional 
relationship between an actual video image captured by a camera and 
a virtual video image when superimposing the actual image and the 
virtual video image on a monitor screen, comprising the steps of: 

capturing actual targets in an actual coordinate system by the 
camera; 

theoretically deriving monitor coordinates in a monitor 
coordinate system on the monitor screen by coordinate conversion of 
actual coordinates of the actual targets in the actual coordinate 
system based on reference values of coordinate conversion parameters 
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including internal parameters of the camera itself and attachment . 
parameters for attaching the camera to the vehicle; 

providing a virtual target in the monitor coordinate system 
on the monitor screen based on the coordinate conversion parameters 
before modification and recognizing the monitor coordinates of the 
image of the actual targets actually captured by the camera based 
on the difference between the monitor coordinate of the image of the 
actual target captured actually by the camera and the corresponding 
monitor coordinate of the virtual target; 

generating relational expressions based on deviations between 
the monitor coordinates of the image of the actual targets and the 
monitor coordinates in the monitor coordinate system of the actual 
targets which have been subjected to coordinate conversion, the 
number of relational expressions being larger than the number of the 
coordinate conversion parameters to be corrected including at least 
internal parameters of the camera itself of the coordinate conversion 
parameters/ 

correcting the coordinate conversion parameters such that the 
square-sum cf the deviations is the minimum; and 

correcting relative positional relationship between the actual 
video image and the virtual video image based on the corrected values 
of the coordinate conversion parameters. 

10 • (cancelled) 

11. A method for the video image positional relationship 

correction according to claim 9, wherein the difference between the 
monitor coordinate of the image of the actual target and the 
corresponding monitor coordinate of the virtual target is calculated 
by moving one of the actual target and the virtual target to 5 position 
overlapped on the other of the actual target and the virtual target 
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on the monitor screen by manipulation of an operator . 

12. A method for the video image positional relationship 
correction according to claim 9, wherein the monitor coordinates of 
the image of the actual targets are recognized by image processing, 

13. (new) A video image positional relationship correction 
apparatus according to claim 1, wherein the recognition means 
automatically recognizes the nearest actual target to the virtual 
target without displaying the virtual target, 

14. (new) A video image positional relationship correction 
apparatus according to claim 1, wherein the correction means 
calculates lines extending between new virtual targets based on the 
coordinate conversion parameters after modification, the lines being 
displayed on the monitor screen. 

15. (new) A method for the video image positional 
relationship correction according to claim 9, wherein the nearest 
actual target to the virtual target is automatically recognized 
without displaying the virtual target. 

16. (new) A method for the video image positional 
relationship correction according to claim 9, wherein lines extending 
between new virtual targets are calculated based on the coordinate 
conversion parameters after modification and displayed on the monitor 
screen . 
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